A neural-network-based investigation of eye-related movements for accurate drowsiness estimation

Many studies reported that eye-related movements, e.g., blank stares, blinking and drooping eyelids, are highly indicative symptoms of drowsiness. However, few researchers have investigated the computational efficacy accounted for drowsiness estimation by these eye-related movements. This paper thus analyzes two typical eye-related movements, i.e., eyelid movements X
_el
(t) and eyeball movements X
_eb
(t), and investigates neural-network-based approaches to model temporal correlations. Specifically, we compare the effectiveness of three combinations of eye-related movements, i.e., [X
_el
(t)], [X
_eb
(t)], and [X
_el
(t),X
_eb
(t)], for drowsiness estimation. Furthermore, we investigate the usefulness of two typical types of neural networks, i.e., CNN-Net and CNN-LSTM-Net, for better drowsiness modeling. The experimental results show that [X
_el
(t),X
_eb
(t)] can achieve a better performance than [X
_el
(t)] for short time drowsiness estimation while [X
_eb
(t)]alone performs worse even than the baseline method (PERCLOS). In addition, we found that CNN-Net are more effective for accurate drowsiness level modeling than CNN-LSTM-Net.